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ABSTRACT 

The Chapter One Children's Art Carnival (C.A.C.) 
Creative Reading Program was designed to improve students' literacy 
skills through participation in creative arts activities integrated 
with instruction in reading and writing. Operating at sites in 
Manhattan, Queens, and the Bronx, the 1988-89 program served 334 
students in second through fifth grade. Each student was involved in 
three types of sessions — an art workshop, a plan and review session, 
and a story room (small group) session. The impact of the program on 
student achievement in reading and writing was determined by 
evaluating students* performance on standardized and norm-referenced 
reading tjsts and holistically scored writing tests against the 
program objectives. Second grade students exceeded the program's 
criteria for success in reading: 97% of the students mastered 3 or 
more skills, 83% mastered 4 or more skills, and 69% mastered 5 or 
more skills. Third grade students did not meet the program's 
criterion for success. All mean differences for the fourth and filth 
grade students wer€> statistically significant and educationally 
meaningful. Students in grades 2, 3, and 4 achieved statistically 
significant mean gains on holistically scored writing tests. Overall, 
52% of the students improved in writing, meeting the program's 
criterion for success. During the past 4 years, approximately 
one-half of the program participants attained five or more reading 
skills and approximately one-half to two- thirds of the program 
participants improved in their writing performance. (Nine tables of 
data are included.) (MG) 
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CHAPTER I CHILDREN'S ART CARNIVAL 
CREATIVE READING PROGRAM, 1988-89 
EVALUATION SUMMARY 

PROGRAM BACKGROUND 

The Children's Art Carnival (C.A.C.) Creative Reading 
Program is a Chapter I program that served 334 elementary school 
students at a total of six sites in Manhattan, Queens, and the 
Bronx in 1988-89. The program was reorganized in the current 
year to include all participants for the full academic year and 
to service two new Bronx schools. The C.A.C. Creative Reading 
Program was designed to provide students with remediation in 
literacy skills through creative arts activities. Program 
activities included art workshops, plan and review sessions, and 
small group reading/writing instruction. 

Student achievement objectives were established by C.A.C. 
staff. The reading achievement of program participants was 
measured differently for second, third, and fourth and fifth 
graders . 



Second graders mastered targeted numbers of skills on 
the Prescriptive Reading Inventory (P.R.I.) and thus 
met the program's criterion for success. 

Third grade student performance did not meet the 
program's criterion for success in reading. The mean 
difference (in Normal Curve Equivalents, N.C.E.s) 
between performance on the spring 1988 Metropolitan 
Achievement Test (MAT) and the spring 1989 Degrees of 
Reading Power test (D.R.P.) was not statistically 
significant. 

♦ Fourth and fifth grade students' performance on the 
D.R.P. met the program's criterion for success in 
reading. The spring 1988 to spring 1989 mean 
differences (in N.C.E.s) were statistically significant 
and represented educationally meaningful effect sizes. 

Tabulation of the number and percentage of students who 
increased in writing proficiency indicates that: 

Fifty-two percent of students participating in the 
program improved, thus meeting program objectives for 
writing performance. 

Comparisons of mean gains in holistir ritJng scores between 
fall 1988 and spring 1989 indicate that: 

overall, C.A.C students achieved a statistically 
significant mean gain in writing. 
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• The mean gains for grades two, three, and four 
represented moderate to large effect sizes. 

Performance during the current year was compared to that of 
previous years. Reading was assessed by the P.R.I, from 1985-86 
to 1987-88; approximately one-half of the student.s attained five 
or more reading r .ills each year, in 1988-89 the P.R.I, was used 
only to assess the performance of second grade students; two- 
thirds mastered five or more additional skills. Comparison of 
holistic writing skills indicated that during the past four years 
approximately one-half to two-thirds of the program participants 
improved their writing performance every year. 

Interviews with artists and the reading teacher at the new 
Bronx sites revealed concerns about scheduling, contact with 
classroom teachers, the need for additional instructional 
materials, and the opportunity for one-on-one work with children. 

Observations of c.A.C. activities indicated two problem 
areas. First, the reorganized format required each member of the 
C.A.C. staff to direct a group of children at virtually all 
times. Secondly, there was a great deal of variation in the 
participation of classroom teachers, site observations also 
revealed that three different ways in which art and literacy 
training were related, including the use of literacy as an 
ancillary to an art project, the use of art as an ancillary to a 
literacy project, and a simple thematic relation between the two. 

Recommendations for program improvement include the 
following: 

The program should establish formal s'taff development 
training sessions. Training should concentrate on the 
integration of art and literacy, the joint pl.mning of 
lessons by artists and reading teachers, and teaching 
artists to teach reading and writing. 

The program should consider flexible scheduling to 
accommodate students who are frequently absent. 
Perhaps these students could be taught individually or 
in small groups on a pull-out basis. 

Program administered testing, possibly with the P.R.I. , 
should be instituted to better assess third grade 
reading performance. 

More detailed direct teaching of writing skills should 
be implemented. This is especially important for 
students with rudimentary writing skills who need to 
learn to write better. 



An explicit {statement of desirable classroom teacher 
participation ^xlOuld be made prior to the beginning of 
the school year; steps should also be taken to ensure 
good participation, classroom teachers should be 
involved with planning themes for the school year, in 
addition, classroom teachers should be briefed before 
each C.A.C. lesson. 

Efforts to increase parental involvement should be 
enhanced. Current efforts should be supplemented with 
parent orientation sessions early in the school year, 
on-going encouragement of parents to visit their 
child's class, and workshops and take-home kits to 
asjsist parent-child reading at home. 

Th€i new site in the Bronx should receive additional 
supplies and equipment so that its facilities are 
comparable to the other sites. Establishment of a 
library of attractive storybooks at this site should be 
a priority. 
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I. INTRODUCTION 



PROGRAM PURPOSE 

The Chapter I Children's Art Carnival (c.A.C.) Creative 
Reading Program is designed to improve students' literacy skills 
through participation in creative arts activities integrated with 
instruction in reading and writing. Secondary goals of the 
program are to facilitate children's abilities to express 
themselves and to motivate them to read and write. 
STUDENT SELECTION CRITERIA 

Students are eligible for Chapter i services if they live in 
a targeted area and score below a designated cut-off point on 
State-mandated standardized reading tests. For grade two, the 
test was the Metropolitan Achievement Test (MAT) , and for grades 
three through six, it was the Degrees of Reading Power (D.R.P.) 
•"^.st. The Chapter I Evaluation Reporting System specifies that 
eligible students may also be selected for Chapter I programs on 
the basis of classroom performance. 
STUDENTS SERVED 

Classes from elementary schools in Community School 
Districts (C.s.D.s) 3 and 5 in Manhattan (P.s, 123M and p.s. 
208M) , C.S.D. 23 in Queens (P.S. 105Q and P.S. 123Q) , and C.S.D. 
8 in the Bronx (P.S. 140X and 146X) participated in the 1988-89 
Children's Art Carnival Program. A total of 334 students in 
grades two through five participated. One-third of the 
participants were in the third grade and one-fourth were in the 
fourth grade 'see Table 1) . 



TABLE 1 



Number and Percentage of Students 
Participating in the Children's Art Carnival' Program 

by Grade, 1988-89 



Grade 


Number 
of students 


Percentage 


2 


66 


19.8 


3 


113 


33.8 


4 


85 


25.4 


5 


58 


17.4 


No grade® 


12 


3.6 


Total 


334 


100. 0 



Special education students are not placed by grade. 

• One-third of the students were in the third 
grade. 

• One-fourth of the students were in the fourth 
grade. 




C.A.C. classes were assigned by the school principal and 
often included problem classes or classes with new teachers. 
During the current year, program participants included a GATES 
class and a MIS I special education class. 

A large number of participating students were holdovers, of 
the 275 participants for whom there was information, 18 percent 
(n-50) were holdovers. Over a third of the students in Queens 
and over a fifth of the students in Manhattan were holdovers; in 
contrast, only eight percent of the students were holdovers in 
the Bronx. Analysis of holdovers by grade indicated that one- 
third of fourth grade students and almost one-quarter of fifth 
grade students for whom there was information were holdovers. 
PROGRAM OBJECTIVES 

Specific student achievement objectives established by 
program staff were as follows: 

• By the end of the 1988-89 school year, 65 percent of 
the students in grade two would master at least three 
additional reading skills on the posttest of the 
Prescriptive Reading Inventory (P.R.I.) that they had 
not mastered on the pretest? 50 percent would master at 
least four additional skills; and 25 percent would 
master five or more additional skills. 

• Students in grades three through five would show 
statistically significant mean gains on standardized 
reading tests. 

• Fifty percent of the students would increase their 
writing proficiency as measured by holistically scored 
writing tests. 
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EVALUATION METHODOLOGY 

Evaluation procedures included on-site observations, 
interviews of program personnel at the new Bronx sites, review of 
prograK documents, and analyses of achievement test results. The 
following questions on the implementation and impact of the 
program were investigated: 

• How is the program implemented? 

• What are the perceptions of C.A.C. staff at the new 
Bronx site regarding the implementation of the program? 

• Were the student achievement objectives for reading and 
writing met? 

SCOPE OF THE REPORT 

This report assesses the implementation and impact of 
the 1988-89 Chapter I Children's Art Carnival Creative Reading 
Program. Program organization and implementation are discussed 
in Chapter II. students' outcome data are analyzed and 
interpreted in Chapter III. Conclusions and recommendations are 
presented in Chapter IV. 
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II. PROGRAM ORGANIZATION AND IMPLEMENTATION 



PROGRAM BACKGROUND AND FUNDING 

The Children's Art Carnival Creative Reading Program hat' 
provided Chapter I eligible students with integrated literacy and 
arts education since 1969. The program operates under the 
direction of the Board of Education's Office of Program and 
Curriculum Development. In 1988-89, the program was funded with a 
grant of $285,000 from the Educational Consolidation and 
Improvement Act (E.G. I. A.) Chapter I, Part A, Basic Grant 
Program . 

Prior to the 1988-89 academic year, central art workshop 
sites in Manhattan and Queens served classes drawn from several 
school districts. The program was reorganized for the 1988-89 
school year so that all program activities were conducted in the 
participating schools. The reorganization entailed conducting 
activities in a smaller numiber of schools than in previous years, 
while serving the same number of students and expanding student 
participation from 18 weeks for most students to a full aceidemic 
year for all participants. Service to two schools in district 8 
the Bronx was also added. 

* 

PROGRAM ORGANIZATION 

Program activities were conducted at sites in Manhattan, 
Queens, and the Bronx, in each of the boroughs, the program 
operated in two schools, one school was a primary site, where 
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throe classes were served. An additional class from another 
school was served in their home school. 

At each site, one reading teacher and two artist-teachers 
provided reading and arts instruction, usually assisted by the 
classroom teacher. Overall, prograw staff included a program 
director, three reading specialists, six artist-teachers, a 
senior educational consultan*;, and an office associate. 

Students participated in the program for a total of two 
hours and 50 iuinutes each week. Each student was involved in 
three types of sessions — an art workshop, a plan and review 
session, and a st .y room (small group) session. These sessions 
were offered either on the same day or on contiguous days. All 
program activities occurred in classrooms designated for program 
use in the participating schools. 
Art Worksho ps 

Art workshops .offered three primary types of creative art 
activities: puppetry, print making, and ceramics. Professional 
artists led the workshop, and classroom and reading teachers 
assisted students. A class of approximately 25 to 30 students 
was divided into two groups and participated in the 90-minute art 
workshop each week. 

Literacy skills and art were integrated with art activities 
through common themes, emphasis on se.quential procedures, and 
reference to printed instructions. Participation in the art 
workshops was intended to complement the development of literacy 
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by encouraging developjnt t of symbolic processes, motivation, 
self -express ion, and self -esteem. 
Story Ro om f Small Gr oup) ^^fisigns 

For each participatiny class, the reading teacher conducted 
severel, usually four, story rooir, (small group) reading sessions. 
These sessions were conducted in a pull-out mode, involving from 
five to seven students; in each session. Students of similar 
ability were grouped together, and an attempt was made to 
individualize instruc':ion,. Each session lasted approximately 40 
minutes. 

Story room activities were related to the art workshops. In 
the small group sessions, stories were read and related to the 
featured theme, and situdents wrote about the theme and about 
their experiences, other activities included reading aloud, 
playing reading gamer (e.g., rhyming, word lotto), and completing 
work sheets using particular reading skills. 
Plan and Re /i ew Sessions 

Plan and review sessions were designed to build on the art 
workshops with instruction in reading and writing skills. The 
reading teacher conducted this session, often with the assistance 
of artist-teacherf nd the classroom teacher. These sessions 
were offered to the entire class and lasted approximately 40 
minutes. 

During plan and review sessions, the staff and children 
discussed options for projects, developed these projects in 
groups, and reviewed their progress. Reading and writing 
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activities were eitiphasizecl. Children kept log books in which 
they recorded work plans and procedures. 
PROGRAM IMP L EMENTAT ION 
Site Observations 

Two observations were conducted at sites in each of the 
three boroughs during the academic year for a total of six site 
observations. At each site,, the OREA evaluator observed the 
three program components — the art workshop, the plan and review 
session, and the story room (small group) session in order to 
assess how the program was being implemented o In addition, the 
Bronx staff was interviewed to discover their perceptions of 
program implementation and their suggestions for program 
improvement . 
Curriculum 

Art Workshops. Art workshops were conceived as pedagogic 
only in a broad sense. They were primarily intended to motivate 
students and allow them an opportunity to experience personal 
accomplishment. At best, they flowed directly from plan and 
review sessions and elaborated on them. 

In one session, the reading teacher announced that the art 
project and the story tney would be reading in the afternoon were 
both about the wi a. Having introduced the theme, the reading 
teacher encouraged students to contribute their ideas. Then, an 
artist demonstrated several examples of toys that twirl in the 
wind and talked about various ways to design them. Finally, 
students cc.npleted work sheets with multiple choice and fill-in- 
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the-blank questions referring to the project, materials used, and 
steps in the process, when most of the students had finished, 
the reading teacher went through the work sheet with them. 

Story Room (Small Groups Sessions > Story room activities 
for small groups of four to six students vjere designed to be 
conducted by reading teachers; observations revealed that on 
several occassions these sessions were also led by artists. Plan 
and review sessions were intended to provide semi-individualized 
reading and writing instruction. 

At one site, an OREA evaluator observed two story room 
sessions that were being conducted concurrently. In one group, 
students read a story aloud, and then the reading teacher a?5ked, 
"Can you think of a better way to tell it?" The students were 
encouraged to write variations. In the other session, one artist 
helped children write short story captions for their art work 
while the other artist was working with a child who needed 
individual attention because of trouble at home. 

At another site, an artist-led session began with a review 
of the group lesson on ecology. Then the artist in; .ated a 
discussion which introduced work on a diorama project. The other 
artist supervised a peer collaborative group doing research on 
the problem of waste disposal, as symbolized by the garbage, 
barge, in preparation for work on their diorama. In a separate 
room, supervised by the reading teacher, stvidents read a story 
out loud. Along the way, the reading teacher asked questions 
about particular words and the situation in the story. 
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Plan ,arid_ R_eyj . ew Sessions . Plan and review sessions are 
intencied to complement art workshop activities. The rationale 
was that pursuing a common theme in both sessions provided a 
motivating context for literacy activities. These sessions were 
condurteci by an artist or a reading teacher. The whole class was 
usually addressed initially, and then two or three small groups 
were £o:naed. 

Sessions focused on either art techniques or themes. Art 
techniques were presented either as preparation for or at the 
conclision of the creation of a piece of art. The discussion 
aftartfards focused on why things were done in a particular way. 
This \>;£is often followed up with students writing in their logs 
about the art project. Sessions included discussion of tools and 
•inaterials, collage, glazing, and overlapping colors. Themes 
included ecology, dinosaurs, the wind, fairy tales, and designing 
a school logo. At their simplest, the sessions involved reading 
a fairy tale aloud. 

The relation between art and literacy varied. In one 
session, for example, literacy was ancillary to the art project. 
A discussion of dinosaur bones and fossils was followed by the 
distribution of step-by-step instructions on how to prepare and 
pour cas-.ts for ersatz fossils. It concluded with students 
preparing and pouring casts. In another session, the* art project 
was ancillary to the literacy project. Artist-led groups created 
stories with the students contributing ideas about setting and 
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characters. Afterwards, they produced corresponding clay models. 
In other sessions, the focus was exclusively on literacy 
activities that were only thematically related to art projects. 
For example, group reading o£ a story about kites was followed by 
a discussion of "wind" words and what could happen to a kite. 
The class concluded with the children writing stories about a 
kite. 

T he Impact of Program Reorganization 

As part of the reorganization of the program, art workshops 
were divided between two classrooms instead of having three 
separate workshops conducted in one room, as in the past. This 
change provided for a less hectic environment. Nevertheless, 
initial site visits revealed that the scheduling of all three 
program components was less than optimal. Informal discussions 
between the OREA evaluator and the program director resulted in 
some modification of the daily schedule to allow for a less 
fragmented day for students and a less frenetic changing of 
activities for the staff. 

The ratio of adults to children was slightly lower than in 
previous years. In the past, a reading teacher and three artists 
had been present for the art workshop and story room sessions. 
The reorganization called for a reading teacher and two artists 
at each site. Furthermore, plan and review sessions replaced the 
whole class story room sessions of previous years. This required 
each adult to direct a group at all times and hindered one-to-one 



attention. Thus, the opportunity for personal connections with 
children has been diminished. 

In several instances, artists were expected to present a 
reading or writing lesson with minimal preparation, while the 
program description called for them to assist a reading teacher. 
The time allocated for daily staff meetings was typically used 
for resolving daily crises, preparing materials for art 
workshops, informally discussing problems with students, and the 
like. The kind of staff development that was needed did not 
occur. 

Finally, there was a great deal of variability in the 
participation of classroom teachers. It was not possible to 
discern whether this variation was due to the individual 
personalities of the teachers or to differences between the 
sites. 

STAFF PERCEPTIONS OF THE PROGRAM 

Since the Bronx sites were newly established, the reading 
teacher and artists at this site were interviewed about the 
implementation of the program. Staff suggestions for 
improvements in the prcgrain were also elicited. 

The reading teacher expressed the need for better 
instructional materials (e.g., story books) and better contact 
with classroom teachers to minimize the duplication of program 
and classroom activities. He also identified a need for more 
time and/or personnel to do provide students with more 
individualized attention. 
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The two artist teachers were interviewed together. They 
expressed concerns about the coordination of the art and reading 
portions of the program. Part of the difficulty was attributed 
to scheduling. At this site, the program had been implemented 
late so that the inclusion of C.A.C. activities in the class 
schedule sometimes resulted in art and reading sessions being 
disconnected in time. In describing how the program was actually 
implemented, they said that there was not enough time for the 
children to complete their art work, talk about it in a group, 
and then write about it in their logs. They noted that plan and 
review sessions could tie things together but they could also be 
redundant with the reading lesson. 

Their suggestions for program improvement included 
allocating more time to meet with the reading teacher to 
coordinate lessons; more preparation time for setting out art 
materials and tools; and more discussion time with students to 
discuss how things learned in the classroom apply to the outside 
world. A final concern was the inclusion of fifth graders in the 
program. These staff members felt they are a different 
population than earlier grades; the program should consider 
beginning remediation earlier, perhaps as early as kindergarten. 
STAFF DEVELOPMENT ACTIVITIES 

The program coordinator had an orientation meeting with each 
participating classroom teacher. Two joint meetings of the 
Manhattan and Queens staff were held during the academic year at 
which staff presentations, discussions, and guest speakers 



focused on improvement of instruction. Program staff also 
attended the annual Rcjasoning Skills Conference conducted by the 
Office of Program and Curriculum Development. 

Staff development sessions were followed up by informal 
discussions throughout the year between reading teachers and 
classroom teachers, in addition, informal weekly meetings of the 
reading teacher and artists at each site concentrated on reading 
and writing instruction. 
PARENTAL INVOLVEMENT 

At the beginning of the school year, the parents of 
participating students received a letter describing the program 
and encouraging them to visit their children's classes and 
accompany them on field trips. In addition, the program 
participated in school open houses in the fall and spring. 
During each open house, an exhibition of the children's artwork 
was supplemented by the staff's discussion of how the program 
integrated art and literacy skills. Finally, the reading teacher 
interacted with the parents' organization at each school. 



ERIC 



14 



III. STUDENT OUTCOMES 

ATTENDANCE 

Mean student attendance in Children's Art Carnival Program 
components was 21.6 days for the art workshop, 19.2 days for the 
story room, and 21.9 days for the plan and review component (see 
Table 2) . Mean rates of attendance in components of the program 
were 72, 59, and 67 percent, respectively. Fifth grade story 
room attendance was especially low. 
METHODS USED TO EVALUATE STUDENT ACHIEVEMENT 

The impact of the program on student achievement in reading 
and writing was determined by evaluating students' performance on 
standardized and norm-referenced reading tests and holistically 
scored writing tests against the program objectives. 

• Students in second grade were expected to master a 
specific number of additional reading skills on the 
posttest of the Prescriptive Readinc Inventory that 
they had not mastered on the pretest: 65 percent would 
master three or more additional skills; 50 percent 
would master four or more; and 25 percent would master 
five or more. 

• For students in grades three through five, the 
objective on standardized reading tests — the 
Metropolitan Achievement Test (MAT) and the Degrees of 
Reading Power Test (D.R.P.)— was that students would 
achieve a statistically significant mean gain from 
pretest to posttest. 

• On the holistically scored writing tests, 50 percent of 
program students were to increase their writing 
proficiency from fall of 1988 to spring 1989. 

Reading Tests 

On the Prescriptive Reading Inventory, tallies were inade of 
the number of reading skills mastered by each student on the 
pretest and posttest, and the number of additional skills 



TABLE 2 

Mean Number and Percentage of Days Present 
in Components of the Children's Art Carnival Program 

by Grade, 1988'-89 



hVt Wpykshpp Story Room Plan & Review 
Number of Mean Mean Mean Mean Mean Mean 
Grade students Days Percent' Days Percent'^ Days Percent'' 



2 


54^ 


20.4 


68.1 


19.9 


61.1 


21. 


8 


67.0 


3 


96 


23.4 


78.0 


21.3 


65.'/ 


23. 


3 


71.6 


4 


81 


20.0 


66.6 


18.7 


57.4 


19. 


9 


61.2 


5 


42 


21.9 


72.8 


14.2 


44.0 


22. 


9 


70.4 


Total 


273 


21.6 


71.9 


19.2 


59.0 


21. 


9 


67.4 


^ The 


mean number 


of scheduled days of 


attendance 


was 


30 


days. 


^ The 


mean number 


of scheduled days of 


attendance 


was 


32. 


5 days. 



For the Story Room component, the total number of students was 



• The mean rate of attendance was 71.9 percent for the 
art workshop, 59 percent for the story room, and 70.4 
percent for the plan and review session. 

• The mean rate of attendance for fifth graders in the 
story room was only 44 percent. 
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mastered by each student on the posttest was calculated. 
Frequency distributions were computed to identify the number of 
students who mastered additional skills. 

On the standardized reading tests, students' raw scores were 
organized by grade and converted to normal curve equivalents 
(N^C.E.s).* Students in grade three took the MAT in spring 1988 
and the D.R.P. in spring 1989. Scores on the D.R.P. (posttest) 
were converted to (pretest) MAT scores so that achievement could 
be measured.** However, since the MAT emphasizes basic skills 
and the D.R.P. emphasizes reading comprehension^, the comparison 
may not be good measure of student achievement. 

Statistical analyses were performed or the N.C.E. scores. 
Correlated £-tests were used to determined whether mean gains 
were statistically significant. Statistical significance 
indicates whether the changes in achievement are real or have 
occurred by chance. However, it can be exaggerated by large 
sample size or depressed by small sample size and does not 
address the issue of v/hether the changes are important to the 
students' educational development. Therefore, an effect size 



*Normal curve equivalent scores are similar to percentile ranks 
but, unlike percentile ranks, are based on an equal- interval 
scale ranging from 1 to 99, with a mean of 50 and a standard 
deviation of approximately of 21. Because N.C.E. scores are 
equally spaced, mathematical and statistical calculations such as 
averages are meaningful; in addition, comparisons of N.C.E. 
scores may be made across different achievement tests. 

** The equi-percentile method was used. T*ie frequency 
distributions of scores on the tests were employed to generate 
conversion tables, which were approved by the state Education 
Department. 

17 

ERIC 2 'V 



(E.S«)* is reported for each mean result to indicate the 
educational meaningfulness of each mean gain or loss. 
Writing Tests 

Student writing samples were obtained at the beginning and 
end of the program year. On both occasions, students were asked 
to draw a picture of a person doing something and then write a 
story about it. 

Writing tests were organized by grade and holistically 
scored using a five point scale that measures how well a student 
communicates. Each writing test was independently scored by two 
readers who were asked to pay special attention to the coherence 
of the passage, its sentence structure and sequence, and the 
writer's awareness of the audience. A discrepancy of one point 
between readers was acceptable. However, discrepancies of more 
than one point required a third reader. 

Inter-rater reliability was computed by comparison of two 
independent ratings of the 494 writing samples. Fifty-three 
percent of the papers received exactly the same score; 42 percent 
received scores that differed by one point. Thus, 95 percent of 
the ratings differed by one point or less, and only five percent 
(24 papers) required a third rater. 



*The effect size, developed by Jacob Cohen, is the ratio of the 
mean gain to the standard deviation of the gain. This ratio 
provides an index of improvement irrespective of the size of the 
samplci. According to Cohen, .2 is a small effect size, .5 is a 
moderate effect size, and .8 is a large effect size. Only effect 
sizes of .8 and above are considered educationally meaningful. 
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After the tests had been scored and converted to N.C.E. 
units, correlated t-tests were used to determine statistical 
significance. Then, an effect size was calculated to determine 
the educational meaningfulness of these mean differences. 
Finally, in order to determine if program objectives for writing 
performance had been attained, the number and percentage of 
. students who had increased in writing proficiency was computed. 
ACADEMIC ACHIEVEMENT IN READING 
Second Grade Students 

Table 3 presents student achievement data on the criterion- 
referenced P.R.I. Students were pretested in fall 1988 and 
posttested in spring 1989. Student achievement was measured 
against the program objectives. Table 3 shows that 

Second grade students exceeded the program's 
criteria for success in redd'^ : 97 percent, 
83 percent, and 65 percent oiPthe students 
mastered three, four, and five or more 
additional skills, respectively. 

Third Grade Students 

Table 4 presents student achievement data on standardized 

reading tests. Third grade students were pretested in spring 

1988 on the MAT and posttested in spring 1989 on the D.R.p. 

Table 4 shows that 

• There was a decrease in test scores from 

pretest to posttest and this difference was 
not statistically significant. Thus, third 
grade students did not meet the program's 
criterion for success. 
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TABLE 3 






Number of Additional Skills Mastered 
on the Pr^^scriptive Readinc,' Inventory 
by Second Grade Students 
in the Children's Art Carnivail Program, j'.988'"89 




Number of Number Percentage 
Skills Mastered of Students " of Students 


2 


1 


3 




5 


14 


4 


5 


14 


5 


8 


23 


6 


8 


23 




o 


17 


8 


1 


3 


9 


1 


3 


Total 


35 


100 


^ Pretest 


and posttest scores were not available for 31 s 


tudents . 


• 


Ninety-seven percent of tha students mastered 
three of wore skills, exceeding the program 
criterion for success. 




• 


Eighty-thre?! percent of the students mastered 
four or more skills, exceeding the program 
criterion for success. 






Sixty-mine percent ol the stud&nts mastered 
five or more skills, exceeding the program 
criterion for success. 
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TABLE 4 



Mean N.C.E. Differences on Standardized Reading Tests 

by Third Grade Students 
in the Children's Art Carnival Program, 1&88-89* 



— Pr^t^St Po sttest Difference Effect 

Grade N Mean S.D. Mean S.D. Mean S.D. Size 

3 76 38.8 1 0.5 37.9 14.8 -0. 9 15.7 -0.1 

® Spring 1989 D.R.P, scores were converted to MAT scores for 
comparison with the spring 1988 MAT. 

• The mean difference was not statistically 
significant. It represented a small effect 



Fourth and Fifth Grade Students 

Table 5 presents student achievement data on the D.R.P. 
Students in fourth and fifth grade were pretested in spring 1988 
and posttested in spring 1989. The new (1987-88) D.R.P. norms 
were used on the spring 1988 and spring 1989 test scores. Mean 
differences were calculated and measured against the program 
objective, a statistically significant mean gain. Table 5 shows 
that 

Students in both grades and overall achieved 
statistically significant mean gains of 11.3 
N.C.E. s. 

Effect sizes were large and represented 
educationally meaningful gains. 

ACADEMIC ACHIEVEMENT IN WRITING 

Table 6 presents student achievement data for grades two 

through five on holistically scored writing tests which were 

administered in the fall and spring of the school year. 

Students in grades two, three, and four achieved 
statistically significant mean gains, thus mecjting the 
program's criteria for success. 

The mean gain of second grade students represented 
an educationally meaningful increase. 

Table 7 presents data on the number and percentage of 

students by grade and overall who increased their proficiency in 

writing. 

Overall, 52 percent of the students improved 
in writing, meeting the progr<im's, criterion 
for success. 

♦ Except for grade four, more than 50 percent 
of the students in each grade iitiprovcd their 
writing skills. 
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TABLE 5 



Mean N.C.E. Differences on the Degrees of Reading Power Test 
by students in the Children's Art Carnival Program 

by Grade, 1988-89' 





Pretest 


Posttest 


Difference 


Effect 
Size 


Grade N 


Mean S.D. 


Mean S.D. 


Mean S.D. 


4 65 


23.1 16.8 


34.4 15.4 


11.3 14.5 


0.8 


5 45 


29.0 19.7 


40.3 17.2 


11.3 14.3 


0.8 


Total 110 


25.5 18.2 


36.8 16.2 


11.3 14.4 


0.8 


®New 1987-88 D.R.P. norms were 
spring 1989 scores. 


used on both 


the spring 1988' and 


^ Mean differences were statistically significant at the 
level . 


P<- 05 


♦ Mean 
were 


differences for 
11.3 N.C.E. s. 


grades four 


and five 





• All mean differences were statistically 
significant and educationally meaningful. 
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TABLE 6 



Mean Raw SCv^re Differences 
on Holistically Scored Writing Tests 
by Students in the Children's Art Carnival Program 

by Grade, 1988-89 



Pretggt Posttest Difference Effect 



Grade 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


Size 


2 


33 


3.1 


1.5 


5.0 


2.2 


1.9" 


1.8 


1.1 


3 


81 


4.4 


1.3 


5.3 


1.4 


0.9" 


1.5 


0.6 


4 


64 


4.7 


1.7 


5.6 


1.5 


0.9" 


1.8 


0.5 


5 


33 


5,2 


4.4 


6.1 


2.3 


0.9 


3.2 


0.3 


No Grade 


^ 6 


2.2 


0.4 


2.7 


1.0 


0.5 


0.8 


0.6 


Total 


217 


4.4 


2.3 


5.4 


1.8 


1.0" 


2.0 


0.5 



The difference was statistically significant at p<.05. 
^ Special education students are not placed by grade. 

• The overall mean gain was 1.0 raw-score 
points. This mean gain was statistically 
significant and represented a moderate effect 
size. 

• The mean gains for grade two, three, and four 
were statistically significant and 
represented moderate to large effect sizes. 
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TABLE 7 



Number and Percentage of Students 
in the Children's Art Carnival Program by <3rade 
with Increased Writing Proficiency, 1988-89 



Grade 


Number 
of Students 


Hiaher Po^tX-^^t Scpy-PR 
Number Percentage 


2 


45 


25 


55.6 


3 


91 


50 


54.9 


4 


76 


35 


46.1 


5 


37 


20 


54. 1 


Total 


249 


130 


52.2 



• More than fifty percent of students overall 
and in grades two, three, and five improved 
their writing proficiency, thereby meeting 
the program objective. 



• Only 46 percent of fourth grade students 
improved their writing skills and thus did 
not meet the program criterion for success. 



Secondary analyses were conducted to determine the pretest 
performance characteristics of students who improved or did not 
improve. The median pretest scores for each grade, computed with 
the combined scores of the two independent raters for each 
student, were two raw-score points in grade two, four raw-score 
points in grade three, five raw-score points in grade four, and 
six raw-score points in grade five. The overall median was four 
raw-score points, scores were partitioned at the median for 
subsequent analyses. 

For grades three, four, and five, more students who scored 
below the median on the pretest improved than did students who 
scored above the median. The pattern for second grade students 
was different. Two-thirds of students with low pretest scores 
did not improve. Those students who did improve were almost 
equally divided between students who had high and low pretest 
scores. This pattern suggests that certain baseline skills are 
necessary for second grade students to benefit from the writing 
practice given. It also suggests that the instruction does not 
help students in higher grades progress beyond a certain point. 
COMPARISON WITH PREVIOUS YFARfi 
Reading T - ists 

Table 8 presents data on reading achievement by students on 
the P.R.I, over the last four school years, 1985-86 through 1988- 
89. However, in 1988-89, the P.R.I, was. used only to assess 
reading performance for students in the second grade, students 
were pretested in the fall and posttested in the spring of the 



TABLE 8 



Number and Percentage of Students 
in the Children's Art Carnival Program 
Mastering the Targeted Numtoer of Skills 
on the Prescriptive Reading Inventory, 1984-85 through 1987-88 







Additional Reatlincf RVnic Maofovori 




Number of 


Three 




Five or More 


Year 


Students 


JS_ 


-1- 


N ^ 


1985-86 


278 


222 


CO 


127 46 


1986-87 


245 


223 


91 


111 45 


1987-88 


279 


235 


84 


145 52 


1988-89 


35" 


34 


97 


24 69 



In 1988-89, the P.R.I, was used only to assess reading 
performance for students in the second grade. In previous 
years, all grades were tested with the P.R.I. 



• From 1985-86 through 1987-88, 80 percent or 
more of the students gained three or more 
reading skills. 

• During this period, approximately one-half of 
the program participants attained five or 
more reading skills. 

• In 1988-89, 97 percent of second grade students 
mastered three or more additional reading skills and 
two-thirds of second grade students mastered five or 
more additional skills. 
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relevant academic year. Results were calculated in terms of the 
percentage of students attaining three or more and five or more 
additional reading skills. Table 8 shows that 

From 1985-86 through 1987-88, 80 percent or more of the 
students gained three or more reading skills. 

During this period, approximately one-half of 
the program participants attained five or 
more reading skills. 

In 1988-89, 97 percent of second grade students 
mastered three or more additional reading skills and 
two-thirds of second grade students mastered five or 
more additional skills. 

t'J riting Tests 

Table 9 presents data on students' writing achievement on 
holistically-scored writing tests from 1986-87 through 1988-89. 
Students were pretested in the fall and po.ttested in the spring 
of the relevant academic year. Results were calculated and 
measured against the program objective that 50 percent of the 
students would increase their writing proficiency. Table 9 shows 
that 

During the past four years, approximately 
one-half to two-thirds of the program 
participants improved in their writing 
performance. 
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TABLE 9 



Number and Percentage of Students 
in the children's Art Carnival Program 
with Increased Writing Proficiency, 
1985-86 through 1988-89 



Year 


Numl^er 
of Students 


Increased pj-oficiencv 
Number Percentage 


1985-86 


222 


106 


48 


1986-87 


185 


123 


66 


1987-88^ 


235 


134 


57 


1988-89 


249 


130 


52 



In 1987-88, a five point rating scale replaced the four point 
scale used in previous years. 



• During the past four years, approximately one-half to 
two-thirds of the program participants improved in 
their writing performance. 



ERIC 



29 

30 



IV. CONCLUSIONS AND RECOMMENDATIONS 

CONCLUSIONS 

During the 1988-89 school year, the Children's Art Carnival 
Creative Reading Program underwent a major reorganization. At 
the new sites in the Bronx, staff interviews revealed a problem 
that is largely attributable to the expansion of the program, an 
unequal distribution of supplies and equipment to the new site. 
In addition, site observations identified two continuing 
problems: uneven participation of classroom teachers and a lack 
of parental involvement in program activities. 

The program's accommodations to budgetary constraints and 
the staff adaptation to a less than optimal schedule were 
laudable. However, scheduling problems made the c.A.C. 
experience appear to be fragmented. This situation improved 
somewhat during the course of the year but was not fully 
resolved. Furthermore, because of both staffing cuts and the 
schedule, the opportunity for personal connections between staff 
and individual children diminished. 

Staff development training sessions could alleviate some of 
these organizational problems. Evaluators felt that staff 
training should prepare artists for the-.r increasingly large 
roles in teaching reading and writing; address the need for 
increasea coordination between artists and reading teachers; and 
the develop ways to better integrate the teaching of art 
literacy skills. Lastly, it must be remembered that alleviation 
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of organizational difficulties is a means to improving student 
literacy skills » 

Attendance emerged as a problem. In the most extreme case, 
the mean number of story room sessions attended by students in 
fifth grade was 14.2 out of a scheduled 32.5 sessions, which 
resulted in an attendance rate of 44 percent. High absenteeism 
is endemic in Chapter I populations, and this undermines the 
effect of remediation sessions. Evaluators felt that efforts 
should be made to reach more students more often. 

Students performed well on tests measuring reading 
achievement. The reading performance of second, fourth, and 
fifth grade students met the program objectives. Fourth and 
fifth grade students achieved statistically significant mean 
gains in their reading scores, and second grade students acquired 
the required number of additional reading skills. However, third 
grade students did not meet the program objective. 

The reading achievement of third grade students decreased 
from pretest to posttest, from 38.8 N.C.E.s to 37.9 N.C.E.s. 
Nevertheless, the mean decrease was not statistically 
significant, and both pretest and posttest scores were above the 
35 N.C.E. threshold for educationally disadvantaged students, in 
addition, these students took the Metropolitan Achievement Test 
(MAT) in spring 1988 and the Degrees of Reading power Test 
(D.R.P.) in spring 1980. since the MAT' emphasizes basic skills 
and the D.R.P. emphasizes reading comprehension, the comparison 
may not be a good measure of student achievement. 



More than fifty percent of students in grades two, three, 
and four and overall improved their writing achievement, li^stting 
the program's objectives. However, program objectives were not 
met by students in grade four as less than half of them improved 
in writing performance. Neverthelesr, , students in grades two, 
three, and four achieved statistically significant mean gains 
from pretest to posttest, and the mean gain of second grade 
students was educationally meaningful. Finally, fifth grade 
students did not achieve a statistically significant mean gain, 
and a large standard deviation indicates that there was a lot of 
diversity in individual student performance. 

Secondary analyses revealed that in grades three, four, and 
five a large proportion of students who entered the program with 
poor writing skills improved, but relatively few students with 
moderate or better incoming skills improved. In second grade, 
this pattern did not hold: two-thirds of students with poor 
incoming skills did not improve. The pattern suggests that 
different kinds of writing instruction may be needed for children 
at different skill levels. 

In conclusion, the 1988-89 academic year should be 
considered a transitional year for the program. Organizational 
and scheduling problems were recognized during the course of the 
year and initial steps were taken to address them. There are 
plans to continue these efforts and expand on them during the 
coming year. 
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RECOMMENDATIONS 

The Bronx sites had faoili.tiew that were not comparable to 
the other sites. Few reading materials were available, there 
were relatively few art supplies, and there was no kiln in which 
to fire ceramics. 

• The Bronx site should receive additional supplies and 
equipment. The establishment of a library of story 
books at this site should be a priority. 

Often, there seemed to be ambiguity about what kind of 

classroom teacher participation was desirable, in addition, 

there is a need for improving tne articulation of program 

activities and regular classroom instruction. 

• Classroom teachers should be involved with the planning 
of themes for the school year. In addition, they 
should be briefed before each program lesson. 

Current efforts to increase parental involvement are 

sporadic. Moreover, these efforts do not guide or support 

parents in assisting their children with reading at home. 

• The program should initiate parent orientation sessions 
early in the school year and encourage parents to visit 
their child's class. In addition, take-home kits for 
parent-child reading at home should be considered. 

There were problems with the integration of art activities 

and literacy skills and the coordination between artists and 

reading teacners. In addition, artists were frequently required 

to teach reading and writing but were given little time or 

support to prepare lessons. 

• The program should establish formal staff development 
trainiiii? sessions. Training should concentrate on the 
integration of art and literacy, the joint planning of 
Wessons by artists and reading teachers, and teaching 
artists to teach reading and writing. 
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Attendance is low for reasons external to the program. Yet 
poor attendance decreases the impact of the program on students 
and contributes to the deflatior^ of student achievement scores. 

• The program should consider flexible scheduling to 
accommodate students who are frequently absent. 
Perhaps these students could be taught individually or 
in small groups on a pull-out basis. 

Third grade students' reading performance decreased from 
pretest to posttest. It is not clear if the decrease was a 
result of using different tests for the pretest and the posttest 
or if it was an accurate measure of students' reading abilities. 
Nevertheless, the program must ensure that its impact on 
students' writing performance is positive. 

• The program should eliminate the use of two separate 
tests to measure the reading achievement of third grade 
students and should consider using the P.R.I. The 
P.R.I, is used as a diagnostic test at the beginning 
of program participation, so only one additional test 
would be required. 

Second grade students with few incoming skills did not 

demonstrate improved writing skills. Students in grades 3, 4, & 

5 with incoming skills better than the average program 

participant in their grade did not improve in writing as much as 

students with less-developed skills. 

•The program must recognize that teaching less-developed 
students to express themselves through writing is a 
different task than teaching more advanced students writing 
skills. 

• More detailed, direct teaching of writing skills should 
be implemented. This is especially important for 
students who are able to write and need to learn to 
write well. 



